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CrUerlon-Referenced Strategks 2 

Crtterton-Referenced Strategies for DlaWsls 

. The traditional method for assessing pupM achievement^ and growth 

has been the use of results from starxdardtzed achtovement tests^. But 

this approach has not proved sa^^factory for several reasons. Subtests that 

might be used d1 agnost leal 1 y generally lack sufficient retiability to 

/permit subtest results to be used in this manner* Co^eqinently, st^tisti' * 

* ■ 

cally rneantngful scores result from larger tests that are globaJ measures 
of reading abMUy rather than discrete ^asures of specific skill at^ 
tainmcnt* Teachers often do notyind these results useful since the 
diagnostic value is limited.^ Furtheftr^re , schools located In low socio- 
economic areas tend to serve ch-il^dren who score lower on standardized 
achievement measures than those attending suburban schools ik) * Are . 
these tests, then, really testing achievement or something else? Ev4&n 
within a school building^ If children are homogeneously grouped, teachers 
of the low ability groups are at a disadvantage if their pupils^ scores 
are compared to those attained by pupils In higher abllitiy groups* Even 
if gain scores are used, children of lower ability are still pen/llzed by 
their slower growth. " Children who speak a. dialect or whose first language 
Is not English^^e at a further disadvantage on these formal testing 

measurev^)'* i" 

Schools, teachers, curricula, etc. seem to have V^fttle impact when 
standardized achievement tests are used as measures'^of pupU Achievement. 
Data from th^ Coleman Report (I), which'were reanalyzed by Hayeskt et al . 
(7)f revedl that only five percent^bf the achievement variance can be 
uniquely attributed to school variables. The Hayeski report' states: 



Crjterlon'Referenced Strategies 3 

f ^ Wl^ all '9rtiups were combined^ and Ra^lal^'Ethnlc Group 
Heint>ershtp was considered as an aspect of fB (Famtty Background), ^ 
as much as halC of the total differences among students tn their 
ACKV {Achievement Index, a composite of achievement test scores) 
could be explained by PB. (p. 117) 

« Therefore, .standardized achievement test scores do not se^"part4cutarly 

useful to the teacher since their results do not reflect specific learnings 

that mlgh^ have been gained during a school year. It was for this reason 

tbat the concept of orl terion^referenced testing was developed. 

First an objective-fs wfitten ^^lA-^etected; the objective should 

— — ' • ^ 

state specif icalty what observable behaviors are expected upon mastery 

of the objective and the conditions Under which the behVvfors are to be 

demonstrated. A criterion performance level, such as StTpercent correct, 

must also be^'stated. An example follows; "Given a list of monosyllabic 

words .each containing a single short vowel sound, the child will pronounce 

the wpr3s with 80^ accuracy." 
O . ^ - 

Next, Items which assess these behaviors are written* The items 

should dlreitty assess what Is stated In the objective. An item, such 

- ^* , ^ 

as meat irf.'the above example, would be unacceptable since It does tiot 
fit the conditions stated in the objecti ve , i .e. , words containing a 
single short vowel sound. Nor would asking th^ child the meaning of a 
word fit the objective since the objective stated only that the child 
should be able to pronounce the word. 

lihe advantages of this type of testing for instruc?^^al planning 
are readily apparent. Th^ teacher knows ejfiactly wh^t the child has and 
has not mastered. For skltls^not mastered the teacher knows whether 
the chlldA^as close to mastery (scoring^ say, 70%) or tn need of much work 
before mastery can be expected (scoring, say» 2t)t). Instruction Is 
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Criterlon'Referenced Strategies k 
* 

efficient since It can be focused speclficaUy on skills that children 
need rather than on skills which certain children may have already 
.mastered A that are taught rn connection with a given story in a basal 
reader. The teacher can compare the diagnostic profiles of a group of 
children; children with a given skill weakness can be grouped together. 
If all teachers of children in a given age range (e.g., six to eight 
years old) assess mastery of children on the same array of skills, then 
teachers, can plan instruction together. One teacher can agree to instruct 
all chtldreh/needlng a gtVen skill i^rhi te others can teach other skills-. 
Teachers can capitalize on their strengths and prepare for Instruction in 
a feAf skills rather than having to offer direct instruction In all skills* 
The children have the benefit oiF improved instruction and a variety of 
teacher's* 

' Some problems may also arise, however. In Implementing the diagnosti^c 
teaching of skills. If teachers exchange pupils for sklH Instructjj 
they must be In constant communication aboiit the progress of each of the 
pupils. Teachers must schedule time for planning meetings to check 
pupil progress and coordinate skill instruction. 8ut perhaps the most 
ijnportant consideration is that skill instruction does not become dn 
Isolated activity so that children may readily trahifer what Is learned 
In skill Instruction to sustained reading* Ih the presentation of new 

J' 

skills, lower level ski t Is shoul d be reviewed. Furthermore, skill In* 
struction should stress application of the skill in context rather than In 
Isolated words. In any k^nd of directed reading lesson skills^ should be 
reviewed and reinforced* Difficulties should l>e noted and corrected. Use 
qC an informaF reading Inventory can be helpful in spotting skills that 
are not being applied In sustained reading as well as checking a child's 
reading level . 
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Criterion-Referenced Strategies 5 

Recently cr i terion^referenced test inglXtS^f^Ing syst^s Jiavje been 
attacked for their fragmentation t)f language processing. The. argument^ 
is that reading !s not learned merely by mastering a series of supposed 
skills. Reading is instead a hatisttc process in which '^there is no 
possible sequencing of skills" (5). Samuels (8) » however » attempts 
to reconcile the holistic and skills approaches by arguing that skill 
, -acquisition is an essential part of learning to read; early instruction^ 

^-<n skills that are conin>on ly recognwtefl can prevait later difficulties 
in reading. 1f» then* one takes the view that skill acquisition is one 
part of^ learning to read as is the assumption underlying most reading 

' p^grams today — t^en criterion-referenced teKing/teaching systems 
xan. provide useful diagf^os£ic information abo^f children's skill mastery* 
What, then* are the important considerations in creating or selecting 
a cfiterij^n-.referenced-testing system? The objectives are, first of ^11 ^ 

4of paramount importance. They must be compati^ble with ttie instructional 
emphases of the school's rea^^^ptogr'am* It is pointless to assess areas 
that are not being taught* lfVth€r objectives are written by teachei^ then 
they are more likely to re^l^ct current reading instruction* If, however, 
a commtfrcial System is to ^e- chosen, the objectives must be carefully 
scrutinized* Do they state the important aspects of instruction? Is 
the sequence of objectives similar to the sequence found in the school's 
instructional program and matjerlals? 

Are the objectives written at a mid-level specificity?^ Objectives 
that are too general (e.g*» the child will read a passage written at 

^ the third grade level with at least 3Z% accuracy \n word recognition) 
~~^e not useful diagnosticdl ly because they do not identify the specific 
diffjcuUies. On the dther hand* objectives should not be so specific 

6 



ERIC 



Criterion-Referenced Strategies 6 

that the reading process does become fragmented and that the results are 
no longer meaningful. When, for example, a system has over four hundred 
oBjVctrves^r tK'e J)Vfmary ^fa^des/ lhan'agismerft t»f IrwtPUCtJon -becon»es a 
colossal task and reading tends to suffer the fate of Humpty Dumpty* if 
objectives are written at a mid-level specificity (e.g., considering 
short vowel sounds as opposed to th^ short a, etc*), then assessrteni 
results can be useful diagnostical 1y. 

A further, very"^ important consideration Is whether technical ^information 
was gathered during test development. Some criterion-referenced testing/ 
teaching systems (e.g., Wisconsin Design for Readinji Skill Development . 9) 
were extensively field-tested and revised before commercial publication. ^ 
Others (e*g.p Foiintal n Val leYLJeacher SuP-Port System In Reading , 2) were 
published without gathering technical data, if a school district plans 

T 

to develop its own crlterion^referenced tests, then It should also plan 
to gather technical data In order to credte effective tests* tf ;4ests are 
not well developed, then the rest of the Additional, supportive components 
(e.g*t a management system, prescriptive teaching resources) are of questionabl 
value. 

Particulary important among technical iJata gathered is test reliability* 
t f objectives are wri tten at a very spec! ti c level , then practi cal I ty woul d 
mandate that only a few items could be used to assess each objective* Test^ 

reliability simply is not adequate with very brief assessments of an item 

♦ 

or^two. Objectives written at mid^level specificity may be assessed by 
more Items (say» 15 or 20), yielding more reliable tests. 

A'final consideration In creating or selecting a cri terlon-^referenced 
testing/teaching system is the type of teacher support desired** Some / 
conmercial systems provide a re'cord-^keeplng system, such as punch cards 
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i 

y^hich are notched whenever a chMd demonstrates mastery of a sktl). / 
Another provides a co^nputen^ printout Indicating Individual skill mastery j 
or deficiency alon^ with prescriptions for Instruction. Other systemsp 
rather than provide specific prescriptions p provide V'esource files which 
suggest a variety of activities and materials from which a teacher may 
choose. In other words, some commercial systems are less prescriptive than others 
rather than placing confidence in the teacher's ability to decide what 
Is the most appropriate and effective instruction for Individual pupils. 



\ 



Conclusion 

CrIt6rion*referenced tests, whether constructed locally or commercially, 
can provide valuable diagnostic information. A teacher can p-lan specific 
skill Instruction, focused on just the skills needed by Individual pupM$. 
if a child has already mastered a skill, skill mastery Is noted, and he/she 
can move on to new skills rather than spending finne on. skills already | 
mastered. CrI ter ion'referenced tests give teachers the- diagnostic , tools 
needed in individualizing, instruction.* The teacher becomes much more i 
effective In Instructional decisions. He/she becomes^ decis lon^^naker j 
rather than a facilitator of directions found in a teacher's manual. 

Criterlon'referenced testing, however, should be supplemented with 



other types of assessment. Standardized achievement tests, while not 



\ 



useful diagnostlcally, do Indicate whetf^r a grx>up of children ^re makln] 
progress in relation to a norm group or aynational standard. Informal 
reading Inventories are diso useful,' Indi^catlng how well a child Is able 
to integrate the various skills In a realistic reading task. Attitudes, 
likewise, should not be forgotten; Knowing how to read is ess^entlalp bu 
the desire to read Is also important. 
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Criterion-Referenced Strategies^ 8 

A final caution, involving' the accountabi lity process, should be 
menttonecj* With cri terion^referenced testing, it Is very easy to deter* 
mine which children, "given instruction in speci f ic ski 1 1 s, have mastered 
those^ skills* It, therefore^ would be easy to conclude that thef teacher 
whose -cMtdren demonstrate mastery of more skills is the more effective 
teacher* While this assumption may be to some extent true, one must 
keep in mind that criterion^referenced tests — ev€tn longer ones ^ ' r 
lack the test reliability of standardized achievement tests* Therefore, 
while criterion-referenced tests may certainly be used to guide instruction, 
only results f rom^ a fairly large groupr of children shouVd.be used in 
teacher or program evaluation (6) . 

' Keeping\ I imitations pf criterion^referenced testing In mind, teachefrs 
may effectively use these tests'to diagnose skill attainment and to provide 
instruction foctised on skill needs* While crIterion-r*eferenced testing 
has been going*6n\ln the classroom for many years, onjy- within about the 



pas^t ten years was ^he concept articulated and systems devTsed to atd We 



teacher in reading instruction. Now an array of convnercial systems exists 
with varying degrees of. teacher support and at differing prices. Teachers 



andiadministrators s'houla first analyze their own instructional program 
^o determine what is neede<^ and then seek or develop a compatible system * 
whicih will prqfvide the most\^sefuI informatfon for local teachers* 



^ Crhtertoh-fteTe^ertted Strategies 9 



ERIC 



References 



t. Coleman, J. S., et^ EquaHty of Educattpnal Opportunity .' Washljigtoh, 
O.C.: U.S. Goverrtment Printing Office, 1366. / ^ 

2. Fountain *Val ley Teacher Support System in Reafllog . Huntl/igtOT Beacft; 

Cal i fornJa: ' Richard L. Zweig Associates, 1971 4 ' 

3. Green, R. L. "Comments on Ebel'^ Defense of Tests and Testing," 

Phi Delta KappaioV 57 (October 1975 a), 88-89. ' ^ 

4. ' 'Green, R. L. "Jtps on Educational Testing; What Teachers & Parents 

■ Should Know,^' Phi ,6elta Kappan . 57 (October 1975 b), 89-93. 

5. Jot^nson, D. D.; and Pearson, P. D. V'Skllls Hanagemen^t Systems: A Crttiqueji^' 

' The Reading Teacher. 28 (8), May 1975, pp. 757r76i*. . 

' ■ " ' ^ ^ ^ . / ' .\ 

6. tiassad, C. "Development and Use of Object We*Refe|^ence3 Te^ts for\ 

Instructional Purposes,'^ Presentation 9t the -First Eastern Regional^ 
Conference of the International Reading Association, 1976.^ 

7. (layeske, G* W., et^-^. A Study of the Achievement' of Our Nation's Students . 

Washingtofv,' D.C.: U.S. Government Printing Office, 1973. 

8. Samuels, 'S. J. "Hierarchical Subskilts in the Reding Acquisitiqn Proce$&." 

Paper presented at the annual' meeting of the international Reading 
^ ' Association, 1975. ^ ; , 

9. Wisconsin Design for Reading SkilT Developnnent . Minneapolis:^ National 
^ , Computer Systems, 1972. ^ ^ 



\ 

\ 



10 



